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Abstract: No formal investigations have been conducted 
into the efficacy or potential influence of reviewer 
recommendations on editorial decisions, and the impact of 
this on the expectations and behaviour of authors, reviewers 
and journal editors. This article addresses key questions about 
this critical aspect of the peer review submission process. We 
suggest several future steps which could be taken towards 
improving the review process and make it more transparent, 
better understood, and fairer for all parties. 

Introduction 

Peer review is the organisational process whereby scholarly 
works are formally scrutinised by experts. This system of 
quality control relies on a division of labour between editors 
and reviewers . 1 However, the roles, responsibilities and 
boundaries between editors and reviewers on the editorial 
decision of a submission are often unclear. Traditionally, 
peer review has been, and is often still, used to assess the 
soundness of manuscripts and to give expert feedback 
for improvements . 2 In many cases, the current journal 
paradigm may also ask for the perceived future impact and 
‘fit’ for the scope of a journal to be considered. 

Problematising the demarcation between editors 
and reviewers, the latter are often also asked to make an 
explicit recommendation to the editor about whether or 


not the work should be published in that venue . 3 Typically, 
recommendations follow the same system as editorial 
decisions, such as (but not limited to) reject, major revisions, 
minor revisions, or accept. There is much variability in this 
process, with some journals communicating this expectation 
clearly as part of the manuscript handling process, others 
requesting any recommendations be accompanied by 
confidential notes to justify them, while others do not 
request recommendations from reviewers at all. 

There are also disciplinary differences, with some fields 
such as computer science relying more on conference 
proceedings as a primary method of communication, 
and in which reviewer recommendations play a much 
more prominent role as they numerically assess whether 
a submission should be rejected or accepted . 4 Due to the 
variability in this process between journals, understanding 
and managing the expectations of different engaged parties is 
critical for a transparent and improved process of peer review. 

Explicit reviewer recommendations can help streamline 
the process through division of labour. They have the potential 
to reduce the relative power and responsibility of the editor 
as the canonical decision maker, or reinforce this power, by 
giving the editor confirmatory recourse or justification. If the 
role of the editor is to align an editorial agenda (i.e., the profile, 
reputation and success of a journal) with a scientific agenda 
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(communication of rigorous research), then distribution 
of the decision-making power (and responsibility) among 
peers is perhaps the more desired ethical process; although 
ultimately it is the editor, or editorial board, who makes 
the final decision. Furthermore, conflating these roles and 
responsibilities could undermine any attempt at determining 
decision making bias and its sources precisely. 5 

This process of reviewers providing recommendations 
creates an unclear boundary between reviewers and editors, 
as there is no clearly demarcated decision-making role. 
Such blurring risks creating a disparity between author 
expectations of a manuscript handling process, and the 
actual process. Despite this clear importance, very little is 
currently understood about the potential impact of such 
recommendations on influencing editorial decisions. 
Questions arise such as what if recommendations are flatly 
ignored, or if they are misused by editors, who blindly rely 
on them to make decisions as opposed to using them as an 
aid to reach their own informed decision? 6,7,8 On the other 
hand, reviewer recommendations could be viewed as too 
harsh, out of scope, or contrary to journal requirements, 
and therefore editors might have a good reason to 
‘ignore’ them. In other cases, reviewer reports might 
include a recommendation, despite there being no formal 
requirement for them. As such, there are many potential 
factors that can play a role in editorial decisions, and 
many of them may not relate to strictly scientific criteria, 
but rather to the vision for the journal and its branding. 
Such values will differ across editors, editorial boards, and 
journals, and reviewer recommendations therefore are part 
of a much more complex decision-making process. 9 This is 
important for two main reasons. Firstly, editorial decisions 
can ultimately make or break the career of academic 
researchers, especially those earlier in their careers. 10 
Secondly, such decisions affect whether or not a manuscript 
is accepted into the peer-reviewed literature. 

Understanding the boundary between editor and 
reviewer 

The key element here is the expected role of the editors and 
reviewers during peer review. Unfortunately, a general lack 
of transparency often obfuscates the processes associated 
with the editorial role, and therefore its impact is difficult to 
empirically analyse. Given that editors and other members 
of editorial boards often have great latitude to decide the 
fate of a manuscript, this lack of transparency is potentially 
problematic. The same goes for any accountability they 
may have towards their publishers. As a result of this, we 
cannot simply assume that editorial judgements are always 
sound. Of course, this does not mean that an editor is 
disallowed from using their own personal judgement to 
make the ultimate decision - that is, after all, an authority 
granted to them as a decision maker. But that there is often 
little accountability for the editorial process, with variable 
requirements for justifying decisions, 11 combined with the 
partial delegation of decision-making to reviewers in a 
way which blurs boundaries of responsibility, means that 
the legitimacy and reliability of the review and editorial 
processes must be called further into question. 


Answering these questions likely hinges on clearly 
defining expectations of reviewers and editors and making 
editorial strategies of reviewer selection more transparent. 
The latter encompasses editors who invite a number of 
reviewers skilled in differing parts of the manuscript 
under consideration. Editors and reviewers may, through 
strategic reviewer choices or careful recommendations, 
attempt to anticipate elements of the synthesis, evaluation, 
normalisation and weighting. Ultimately, we can ask which 
elements of the editorial judgements are facilitated by inviting 
reviewer recommendations. Providing more transparency 
here should help to make the process more fair, verifiable, 
and trustworthy 

Key questions and discussion 

Key questions arising from this, which are highly contingent 
on (inter)disciplinary community norms, are: 

1. Should journals invite reviewer recommendations, 
either built into manuscript handling systems or 
within the reviewer reports? 

2. Should such recommendations be mandatory or 
optional? 

3. Do recommendations form part of reviewer best 
practices? 

4. Should authors see these recommendations? 

5. Should the factors upon which the recommendations 
are based be clearly stated across different venues (eg 
novelty, perceived impact, quality)? 

6. What is the expected distribution of responsibilities 
and tasks between reviewers and editors, and how 
does this vary across communities? 

7. What is the impact of these recommendations on 
editorial decisions? 

What reviewer recommendations (to editors) do 

We can articulate a series of informed expectations both 
in favour and against such inclusions and will, here, develop 
them further. 

We hypothesise the five most salient expectations of 
reviewer recommendations as follows: 

1. We may expect that inviting recommendations would 
more evenly distribute responsibility for the ultimate 
editorial decision on publication, often across reviewers 
with a diverse range of opinions, review styles, and 
experiences. This should enable a more complete 
expertise palette to contribute to judgements, as well as 
a more variable value palette to counteract undesirable 
normative evaluations. 

2. It signals to reviewers that editors and journals 
trust their expertise and are willing to take their 
considerations seriously. 

3. It has the potential to help editors make a more clear 
and legitimate decision, especially if they are under 
pressure to handle high volumes of papers or oversee 
a broad expertise area they cannot possibly cover by 
themselves. However, contradictory recommendations 
are likely to hinder this advantage, making the weighing 
or synthesis of evaluations difficult. 

4. More critically, it runs the risk of becoming a proxy 
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for a more thorough evaluation, a numerical input in 
a decision algorithm. 

5. It allows editors to tease out for themselves how reviewers 
think without having to either filter-out or factor-in 
relevant cultural differences in potential decisions; for 
example, regarding language preferences, seniority or 
status, institutional affiliation, gender, or region-specific 
significance. 

Before we can discuss the proportional contribution of 
all these expectations to a final position, the status of the 
recommendation offered by reviewers needs to be considered. 
Making the recommendation mandatory signals that editors 
might be unable to reach conclusions themselves. Reviewers 
may have good reasons to withhold judgement. For instance, 
a reviewer may be capable of reviewing all the epistemic 
claims of a paper with a keen eye for detail, but unable to fully 
assess the statistical methods used; which some manuscript 
handling systems ask reviewers to be explicit about. That 
reviewer would usually indicate the limits of their review (or 
would receive a question classifying their competence in the 
matter) and given such limitations, no sensible judgement 
can be made. 

Making the recommendation voluntary could mean that 
some reviewers choose to withhold judgment for other reasons 
than fair assessment of the paper - although they could also 
opt for a judgement that mismatches the content of the paper 
- if they have an interest in the paper being published/ rejected. 
Either way, non-mandatory recommendations within a 
journal could potentially introduce further inconsistencies 
into the decision-making process. 

While the status of the recommendations could be written 
down in editorial policies, a lot of leeway is expected to 
remain in the daily operations of any publishing practice. 
Recommendations could be just that: a piece of advice, to be 
treated as the editor pleases. However, the recommendation 
could also carry more power in the sense that the editor would 
have to follow it, except when they could argue how and why 
they chose not to; in cases where this happens, transparency 
is clearly required to justify conflicting decisions to both 
reviewers and authors. 

The status of the recommendation tends to be very unclear, 
meaning that when reviewers issue a recommendation, they 
cannot know how it will be treated. Reviewers can disagree on 
their recommendation. They do so, in fact, regularly. While 
reviewers are, in the current system, not pressured to reach 
consensus (ie they have no synthetic duty), the editor will have 
to reach a single decision - the synthetic duty lies with them. 

Reviewer recommendations in practice: 
transparency, variability and responsibility 

Scientific publishing is rapidly becoming more transparent 
in a multiplicity of dimensions, ranging from open review 12 
and open data 13 to open access publishing 14 . However, this 
does not mean that all decision processes are being made 
public, and indeed this aspect is strikingly absent from most 
recent discussions of open review, with some exceptions, 15 
including the Peer Review Evaluation initiative 16 or 
the TRANsparency in Scholarly Publishing for Open 


Scholarship Evolution (TRANSPOSE) project. 17 

Open review, or open science in general, is currently 
not typically accompanied by open editorial conversation 
or open legitimations of editorial decisions; although there 
are some exceptions, such as the journals PeerJ and BMJ 
with optionally public editorial processes that include 
reviewer recommendations. To this seeming lack of editorial 
accountability; how do recommendations by reviewers 
contribute to holding editors accountable? Providing 
recommendations as part of an (open) peer review allows 
either public and authors or, in the case of closed review the 
authors themselves, to use them as arguments to appeal, or 
question editorial decisions. 

This does not mean that there is any duty for editors to 
uncritically adopt reviewer recommendations. Editors have 
to consider a lot of competing valuation systems. 18,19 ’ 20 ' 21 
Journals range from very small to very large, both in terms 
of the number manuscripts they handle and publish, as well 
as in terms of their epistemic scope. Some are commercial 
entities while others are run by volunteers. Some are 
ideologically committed towards open science while 
others are more hesitant. These variable profiles translate 
into different relationships to their reviewers and authors, 
and diverse expectations of those who interact with the 
journal. They will most likely shape expectations of editors 
overseeing such interactions and as a consequence, confront 
each editorial office with a unique balance of diverse value 
systems. This also raises the question of whether the 
recommendation by the reviewer to the editor ought to 
have a conditional status, since the editor can draw from 
other valuation systems to overrule it. 22 

Ultimately this paper invites the question over to what 
extent the wider research community can, and should, trust 
editors with this responsibility and power. Transparency does 
not, in itself, enhance trust or even trustworthiness. While 
trust is a characteristic in a relationship, trustworthiness is 
a characteristic of an individual or institution. Importantly, 
it can be enhanced and maintained via optimisation of 
its key ingredients: competence, reliability and honesty. 23 
Trustworthy editors and reviewers are competent, reliable 
and honest and through repeated displays of all three, can 
build credibility; the goal of the consensus statement on core 
competencies for editors includes ensuring these traits are 
present. 24 

Recommendations 

We recognise that peer review can never be a ‘perfect’ process. 
However, there are certain steps that can be taken towards 
improving the process, making it more transparent, better 
understood, and fairer for editors, reviewers, and authors. 
With respect to reviewer recommendations, these include: 

• For journals to share data on the peer review workflows 
and decisions made by editors, and the respective 
recommendations from reviewers and... 

• ...gain insight into the user experiences of manuscript 
handling systems. We could analyse the effects and use 
the information to improve review and editorial systems. 

• For journal policies tobe clear to all parties about the current 
expectations and roles of reviewers and editors, especially if 
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these policies change. Here, greater transparency about the 
division of labour and responsibility is key to managing the 
process and could help to alleviate some of the confusion 
around the roles of different parties involved. 

• Greater transparency for readers regarding editorial 
decisions, for example, by widening the practice of naming 
the editor responsible for accepting a manuscript, along 
with disclosing reviewer recommendations (even where 
review reports or reviewer names are not made public). 
These suggestions do not represent all possible future 
ways forward, nor do we expect them to be agreed upon by 
all communities and stakeholder groups. However, we hope 
that this article catalyses scholarly communities into further 
discussion on this important aspect of the peer review 
process, and we can begin to gain some empirical insight 
into the potential impacts of reviewer recommendations. 
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